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Abstract 

Background: Spinal cord injury (SCI) can leave the affected individuals with paraparesis or paraplegia, thus 
rendering them unable to ambulate. Since there are currently no restorative treatments for this population, novel 
approaches such as brain-controlled prostheses have been sought. Our recent studies show that a brain-computer 
interface (BCI) can be used to control ambulation within a virtual reality environment (VRE), suggesting that 
a BCI-controlled lower extremity prosthesis for ambulation may be feasible. However, the operability of our BCI 
has not yet been tested in a SCI population. 

Methods: Five subjects with paraplegia or tetraplegia due to SCI underwent a 10-min training session in which 
they alternated between kinesthetic motor imagery (KMI) of idling and walking while their electroencephalogram 
(EEC) were recorded. Subjects then performed a goal-oriented online task, where they utilized KMI to control the 
linear ambulation of an avatar while making 10 sequential stops at designated points within the VRE. Multiple 
online trials were performed in a single day, and this procedure was repeated across 5 experimental days. 
Results: Classification accuracy of idling and walking was estimated offline and ranged from 60.5% (p=0.0176) to 
92.3% (p=l. 36x10"^°) across subjects and days. Offline analysis revealed that the activation of mid-frontal areas 
mostly in the /i and low /3 bands was the most consistent feature for differentiating between idling and walking 
KMI. In the online task, subjects achieved an average performance of 7.4±2.3 successful stops in 273±51 sec. 
These performances were purposeful, i.e. significantly different from the random walk Monte Carlo simulations 
(p<0.01), and all but one subject achieved purposeful control within the first day of the experiments. Finally, all 
subjects were able to maintain purposeful control throughout the study, and their online performances improved 
over time. 

Conclusions: The results of this study demonstrate that SCI subjects can purposefully operate a self-paced BCI 
walking simulator to complete a goal-oriented ambulation task. The operation of the proposed BCI system requires 
short training, is intuitive, and robust against subject-to-subject and day-to-day neurophysiological variations. 
These findings indicate that BCI-controlled lower extremity prostheses for gait rehabilitation or restoration after 
SCI may be feasible in the future. 
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Introduction 

Spinal cord injury (SCI) can leave the affected individuals with paraparesis or paraplegia, thus rendering 
them unable to ambulate. Since there are currently no restorative treatments for this population, techno- 
logical approaches have been sought to substitute for the lost motor functions. Examples include robotic 
exoskeletons 1|, functional electrical stimulation (FES) systems [2], and spinal cord stimulators However, 
these systems lack the able-body-like supraspinal control, and so the ambulation function of these devices 
is controlled manually. In addition to being unintuitive, these systems may be costly and cumbersome to 
use, and therefore have not yet garnered popular appeal and adoption among potential SCI users. Due to 
these limitations, wheelchairs remain the primary means of mobility after SCI. Unfortunately, the extended 
reliance on wheelchairs typically lead to a wide variety of comorbidities that constitute the bulk of chronic 
SCI-related medical care costs (4||7]. Consequently, to address the above issues associated with the treatment 
of paraparesis and paraplegia after SCI, novel brain-controlled prostheses are currently being pursued [8j|9] . 

Recent results by our group [8|[9] suggest that an electroencephalogram (EEG) based brain-computer 
interface (BCI) controlled lower extremity prosthesis may be feasible. More specifically, these studies demon- 
strated the successful implementation of a BCI system that controls the ambulation of an avatar (a stand-in 
for a lower extremity prosthesis) within a virtual reality environment (VRE). By using a data-driven ma- 
chine learning approach to decode the users' kinesthetic motor imageries (KMIs), this BCI-controUed walking 
simulator enabled a small group of subjects (one with paraplegia due to SCI |9 ) to achieve intuitive and 
purposeful BCI control after a short training session. While the single SCI subject outperformed most able- 
bodied subjects in this study, the operability of this system has not yet been tested in a SCI population. The 
successful implementation of the BCI-controUed walking simulator in a population of subjects with SCI will 
establish the feasibility of future BCI-controlled lower extremity prostheses and will represent an important 
step toward developing novel gait rehabilitation strategies for SCI. 

Extending the application of the BCI-controUed walking simulator to a SCI population is faced with 



several problems. First, cortical reorganization, which is common after SCI 10-13 , may cause the cortical 



representation of walking KMI to vary vastly from one SCI subject to another. Second, this representation 



may dramatically evolve over time when SCI subjects are engaged in KMI training 14 . Finally, subjects 
with SCI may interpret walking KMI either as motor imagery or as attempted walking, which in turn may 
result in multiple patterns of cortical activation across these individuals. Therefore, intuitive BCI operation 
under these conditions requires a system that can accommodate for the variations of brain physiology across 
SCI individuals, time, and strategies. To address these problems, we used a data-driven machine learning 
method to decode walking KMIs in a small population of SCI individuals. This approach enabled 5 subjects 
to achieve intuitive and self-paced operation of the BCI-controUed walking simulator after only minimal 
training. Furthermore, they were able to maintain this level of control over the course of several weeks. 



Methods 
Overview 

The goal of this study was to determine if individuals with complete motor SCI can use intuitive control 
strategies to purposefully operate a BCI-controUed walking simulator. To achieve this goal, 5 subjects with 
SCI underwent a short training procedure where they performed alternating epochs of idling and walking 
KMI while their EEG were recorded. These training EEG data were then analyzed to build decoding models 
for subsequent online BCI operation. To ascertain purposeful BCI control, subjects then performed 5 sessions 
of an online BCI goal-oriented virtual walking task [9*. This entire procedure was performed 5 times over 
the course of several weeks to determine if subjects' performances improved with additional practice. 



Subject Recruitment 

This study was approved by the University of California, Irvine Institutional Review Board. Four subjects 
with paraplegia and one with tetraplegia due to SCI were recruited via physician referral from the Long 
Beach Veterans Affairs Spinal Cord Injury Center and other SCI outreach programs. The subjects (see 
Table 1) gave their informed consent to participate in the study. Note that all subjects were BCI naive and 
most of them performed the experimental procedures at a rate of once per week. 
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Subject Gender Age SCI status 



1 


M 


34 


Til, ASIA A, 8 yr 


post injury 


2 


M 


46 


Tl, ASIA B, 4 yr. 


post injury 


3 


M 


43 


C5, Syringomyelia, 


14 yr. post onset 


4 


M 


59 


Tl, ASIA B, 2 yr. 


post injury 


5 


M 


21 


Til, ASIA B, 1 yr 


post injury 



Table 1: List of participants with demographic data and level of SCI. ASIA = American Spinal Injury 
Association impairment scale. 



Data Acquisition 

Each subject was positioned ~0.8 to 1 m from a computer screen that displayed textual cues or the VRE. EEC 
were recorded using a 63-channel cap (Medi Factory, Heerlen, The Netherlands) with Ag-AgCl electrodes 
arranged according to the extended 10-20 International Standard. Conductive gel (Compumedics USA, 
Charlotte, NC) was applied to all electrodes and the 30-IIz impedances between each electrode and the 
reference electrode were maintained at <10 Kfi by abrading the scalp with a blunt needle. The EEC 
signals were amplified and digitized (sampling rate: 256 Hz, resolution: 22 bits) using two linked NeXus-32 
EEG bioamplifiers (MindMedia, Roermond-Herten, The Netherlands). These signals were then streamed 
in real-time to a computer, re-referenced in a common average mode, band-pass filtered (0.01-40 Hz), and 
subsequently analyzed. The above procedures were performed using custom-written MATLAB (MathWorks, 
Natick, MA) programs. 



Training Procedure 

To facilitate intuitive control of ambulation within in a VRE, a EEG decoding model was developed that 
differentiates between idling and walking KMI. To this end, training EEG data were acquired while sub- 
jects underwent 30-sec alternating epochs of idling and walking KMI over a 10-min session. Subjects were 
instructed to perform walking KMI and idling via automated textual cues displayed on the screen. For the 
walking KMI task, subjects were coached to vividly imagine themselves walking or to attempt to perform 
the cyclic (albeit ineffective) leg movements of walking. During this entire procedure, a computer labeling 
signal (to identify idling and walking KMI epochs) was recorded by an auxiliary data acquisition system 
(MP150, Biopac Systems, Goleta, CA), and a common pulse train sent to both EEG and auxiliary data 
systems was used to synchronize the EEG and labeling signals. Also, the subjects were asked to remain 
still during the entire procedure, and their movement was monitored by the experimenter. If consistent 
movements were observed, the subject was asked to repeat the entire training procedure without making the 
undesired movements. 



Decoding Model Generation 

Offline signal analysis to generate EEG decoding models was performed in a manner similar to Wang et al. [o] . 
Briefly, EEG and labeling signals were first aligned and merged using the synchronization signal, and an 



iterative artifact rejection algorithm 15 was used to exclude EEG channels with excessive electromyogram 
(EMG) artifacts from further analysis. The pre-processed EEG data were then split into segments of idling 
and walking KMI states using the labeling signal. Each EEG segment was then divided into 5 randomly 
spaced, non-overlapping 4-sec trials, which were then transformed into the frequency domain using the Fast 
Fourier Transform (EFT). Note that the remaining data in each segment were discarded. Finally, the power 
spectral density of each trial was integrated in 2-Hz bins, resulting in 20 binned power spectral values per 
channel. 

Once the EEG segments were transformed into the frequency domain, a systematic search was performed 
to find the best contiguous frequency range for classification [9]. The dimension of the input data was 



then reduced using classwise principal component analysis (CPCA) [16 17 , and discriminating features 
were extracted using either Fisher's linear discriminant analysis (LDA) [18| or approximate information 
discriminant analysis (AIDA) [l9p0 . This resulted in the extraction of one-dimensional (ID) spatio-spectral 



features: 



/ = T*c(d) (1) 
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where d e M^^*-^ is a single-trial of EEG data with B frequency bins per channel and C retained EEG 
channels, $c ■ K^^*^ — > IR™ is a piecewise linear mapping from the data space to an m-dimensional CPCA- 
subspace, and T : M™ ^ M is a LDA or AIDA transformation matrix. 

Once the single-trial data were reduced to ID spatio-spectral features, a linear Bayesian classifier: 

^.^f^, if Pii\n>p{w\n 

I W, otherwise 

was designed in the feature domain, where P{I |/*) and P(W |/*) are the posterior probabilities of idling and 
walking classes given the observed feature, /*, respectively. Eq. Q can be read as: "classify /* as idle class 
if the posterior probability of idling is greater than the posterior probability of walking given the feature, /*, 
and vice versa." The offline performance of the Bayesian classifier ([2]), expressed as a classification accuracy, 
was then assessed by performing 10 rims of stratified 10- fold cross-validation (CV) [2l]. To determine the 
optimal parameters for classification, the lower bound of the frequency range was then increased in 2-Hz 
steps, and this procedure was repeated until the classifier performance stopped improving, allowing the 
optimal lower frequency bound to be determined. The upper bound of the optimal frequency range was 
determined in a similar manner. The classification accuracy was also used to decide whether to use AIDA or 
LDA as a feature extraction technique. Finally, the parameters of the decoding model, including the optimal 
frequency range, the feature extraction mapping, and the classifier parameters, were saved for real-time EEG 
analysis. 



Online Signal Analysis 

During online operation, 0.5-sec long segments of EEG data were acquired in real time (at a rate of two 



segments per second), and were processed using the methods described in the Decoding Model Generation 



section. Briefly, the EEG signals were first band-pass filtered, followed by automated artifact rejection. 
The pre-processed EEG data were then transformed into the frequency domain using EFT, and the power 
spectral densities over the optimal frequency range were calculated. These spectral data were used as an 
input for the feature extraction algorithm ([T]). Finally, using Bayes' rule the posterior probabilities of 
idling and walking classes given the observed EEG feature, /*, were calculated. 



Online Calibration 

Since the BCI system is a binary state machine with an idling and a walking state, a brief calibration 
procedure was required to identify the posterior probability thresholds to transition between these two 
states [9]. Using the decoding model developed during the training procedure, the BCI system was set to 
run in the online mode while subjects were prompted to alternate between performing idling and walking KMI 
for ~2 min. The posterior probabilities were recorded and their histograms were plotted to determine two 
thresholds, one to initiate the ambulation of the avatar, Tyy , and the other to stop the avatar, Tj. These two 
thresholds were initially set as Tw = median {P(W \f* € W)} and Tj = median {P(yV |/* G I)}. Then, a 
brief online test was performed and based on the subject's feedback, the thresholds were adjusted as necessary. 
During online BCI operation, the posterior probabilities P(W |/*) corresponding to the most recent 1.5 sec 
of EEG data were averaged and compared to the thresholds Tyy and Tx, and the state transitions were 
executed based on the finite state machine diagram given in Fig. 1. This step was implemented in order to 
smooth the sequence of posterior probabilities and reduce noisy state transitions. 



BCI and Virtual Reality Environment Integration 

Using a simulated physics environment (Half-Life 2 Garry's Mod, Valve Corporation, Bellevue, WA), a VRE 
was constructed to represent a fiat grassland with 10 non-player characters (NPCs) arranged in a linear path 
in front of an avatar (see Fig. 2). During online evaluation, the subjects were assigned the task of utilizing 
idling and walking KMI to control the avatar's standing and walking, similar to (8||9]. A virtual joystick 
program [22] was used to interface the BCI software (MATLAB) and the VRE to enable the BCI's control 
of the avatar's walking. Full details on this integration process can be found in Wang et al. j9j. 
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Figure 1: The BCI system as a binary state machine with walking and idhng states represented by circles. 
The state transitions are represented by arrows, with transitions triggered by the conditions shown next to 
the arrows. Self-pointing arrows denote that the system remains in the present state. 




Figure 2: The avatar controlled by the BCI within the VRE. The simulator is operated in 3'^'^ person view. 
The subject uses walking KMI to move the avatar to each NPC and idling KMI to dwell there for at least 2 
sec. 
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Online Performance and Assessment 

To assess the online BCI performance, subjects were tasked to move the avatar within two body lengths of 
each NPC and remain idle at each location for at least 2 sec. On each experimental day, subjects repeated 
this task for 2-8 online sessions. This depended upon the subject's willingness and availability. In total, 
each subject underwent between 19 and 29 online sessions performed over 5 experimental days conducted 
over the course of several weeks. Two performance measures were recorded for each session: the number of 
successful stops at the NPCs and the time taken to complete the course. The successful stops were defined 
as follows. Subjects were given one point for dwelling at the designated stop for at least 2 sec, and a fraction 
of a point for dwelling at the designated stop for at least 0.5 sec. No point penalty was incurred for dwelling 
longer than 2 sec, but this inherently increased the time to completion. Thus, the maximum successful stop 
score was 10 points. In addition, subjects were given a 20-min time limit to complete the course. If the 
subject could not finish the course within 20 min, the trial was interrupted and the number of successful 
stops achieved thus far was recorded. 



Control Experiments 

The online performances (number of successful stops and completion time) were compared to random walk 
to determine if purposeful control was attained. The random walk performance was simulated by uniformly 
sampling the posterior probabilities between and 1 and following the state transition rules with threshold 
values presented in Online Calibration] of the Methods section (details can be found in 9|). To ensure 
statistical significance, 1000 Monte Carlo (MC) simulation runs were performed and the number of successful 
stops and completion times were logged (the 20-min completion time limit was enforced as above). In 
addition, the same task was performed by an able-bodied subject using a manually controlled physical 
joystick [^. 



Statistical Tests 

Purposeful control was defined as the ability to complete an online session within the allotted 20 min, 
with performances significantly different from random walk (p<0.01). This was ascertained by comparing 
the subjects' online performances to MC simulations using a multivariate analysis. To this end, the 2D 
probability density function (PDF) of each subject's simulated random walk performance (consisting of 
two variables: number of successful stops and completion time) was estimated using the Parzen-Rosenblatt 



window method 18 23 . A constant- value PDF contour was then drawn through each subject's observed 
performance point and the corresponding p-value was determined as the fraction of the volume under the 
random walk PDF outside of this contour. These p- values were used to ascertain a statistically significant 
difference between the online performances and the simulated random walk. Finally, subjects are deemed 
to have attained purposeful control once they have a single online session that is purposeful based on the 
definition above. 



Composite Score 

Given the difficulty of interpreting multivariate performance measures across subjects and sessions, a single 
composite score was defined as the following geometric mean: 

C = y/CsCt (3) 

where Cs and Ct are the normalized performance measures for the number of successful stops and completion 
times, respectively, i.e. 



t -t 



Ct 



Here, s is the subject's number of successful stops, Smax = 10 is the maximum number of successful stops, 
t is the subject's completion time, tmax = 1200 sec is the maximum allowed time, and tj^m = 201.52 sec is 
the minimum time required to complete the course while achieving 10 successful stops. The values of and 
Ct, and consequently c, range from to 100%, where 100% corresponds to a perfect performance. Note that 
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the use of the geometric mean favors a performance that is balanced over a performance that sacrifices one 
performance measure over the other (e.g. when a subject finishes the course in a short time while failing 
to make stops). Also note that the normalization of Cg and Ct ensures that the performance measures are 
unitless. 



Results 

Offline Performances 

After subjects underwent the training data collection for each experimental day (see Training Procedure 



tion), subject-specific EEG decoding models were generated as described in the Decoding Model Generation 



section. Cross-validation of these models resulted in offline classification accuracies that ranged from 60.5% 
to 92.3% (Table 2), with corresponding p-values between 0.018 and 10~^°, thus indicating that the classifier 
performances were well above the chance level (50%). In addition, the subjects' average offline performances 
over 5 experimental days were 82.3%, 71.8%, 82.3%, 82.5%, and 82.2%, respectively. Finally, the overall 
average for all 5 subjects across all 5 experimental days was 80.2% ± 8.62 (n — 25). 

Spatio-spectral feature extraction maps generated from offline analysis, as described in the |Decoding| 
[Model Generation! section, revealed salient brain areas and frequency bands underlying idling and walking 
KMI for each subject. Qualitative analysis of these topographic maps revealed significant variations in the 
brain areas utilized by the subjects while performing this task. For example, Subject 2 used mostly the 
Cz area (Fig. 3), whereas Subject 5 used areas C3 and C4 (Fig. 4). Furthermore, the utilized brain areas 
and frequency bands evolved for each subject over the 5 experimental days. For example. Subject 2 had 
growing activation of mid-frontal areas up to experimental day 4, followed by a slight shrinkage on day 5. 
However, one consistent feature among all subjects was the activation of mid-frontal areas (over Cz or FCz 
and adjacent electrodes) mostly in the /i (8-12 Hz) and low f3 (12-16 Hz) bands. 



Online Calibration 

After the short calibration procedure (see Online Calibration in the Methods section), the state transition 
thresholds, Tx and Tyy, were determined from the distributions of the posterior probabilities, P{W |/*) (see 
Fig. 5 for an example), and their values are presented in Table 3. Note that in an ideal situation, P(W |/* G 
W) and P{yV\f* & T) should cluster around 1 and 0, respectively, but as long as these probabilities are 
separable, online BCI control should be achievable. The values of Ti and Tyy ranged from 0.07 to 0.70 and 
from 0.09 to 0.90, respectively, and the average across all subjects on all experimental days was 0.40 and 
0.62 for Tx and Tw, respectively. 



Online Performances 

All subjects were able to achieve purposeful online control immediately (on day 1), with the exception of 
Subject 2 (on day 2). The online BCI performances upon achieving purposeful control are summarized in 
Table 4. The average stop score was 7.4 ± 2.3, and the average completion time was 273 ± 51 sec across 
all online sessions (n = 124). Note that minimal lapses in BCI control occurred, as only 5 out of 124 online 
sessions (~4%) were non-purposeful. Examples of the best online performances for each subject are shown 
in Fig. 6 along with the simulated random walk PDFs. For additional comparison, the average performance 
achieved by a manually controlled physical joystick was 9.38 ± 0.95, and on their best day. Subjects 3, 4, 
and 5, achieved a similar number of successful stops; however, no subjects were able to complete the course 
as fast as manual control. A representative time-space course of an online session (see Fig. 7) shows only a 
single false alarm and a single omission. Also, a supplementary video showing a representative online session 
from Subject 3 is can be found at http://www.youtube.com/watch?v=K4Frq9pwAz8, 

The composite scores were calculated from the online performance measures and are summarized in Table 
5. In general, the performances improved significantly over time; the average on day 1 was 77.8% and the 
average on day 5 was 85.7% (p=0.0302). For comparison, the composite score of the joystick task is also 
shown. On their best days. Subjects 3, 4 and 5 achieved performances similar to those of the manually 
controlled joystick, reaching nearly perfect performances (100%). 
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Offline Performances 



Subject 


Day 


r^(correct 1/ j (/o) 


p- value 


1 


1 


71.9 ± 2.2 


6.29 X IQ-^ 




2 


89.4 ± 1.2 


1.53 X 10"^'^ 




3 


83.9 ± 2.0 


1.30 X 10-12 




4 


84.0 ± 1.9 


1.30 X 10-12 




5 


82.2 ± 1.7 


6.55 X 10-12 




Avg. 


oz.o ± D.4 


i.ZO X iU 


2 


1 


62.2 ± 1.8 


6.00 X 10-^ 




2 


62.0 ± 1.8 


1.05 X 10-2 




3 


60.5 ± 2.0 


1.76 X 10-2 




4 


91.6 ± 1.7 


1.60 X 10-19 




5 


82.5 ± 1.6 


6.55 X 10-12 




Avg. 


71 o 1 1 ^ o 
I i.O ± 14. d 


D.oz X iU 


3 


1 


90.3 ± 1.3 


1.66 X 10-1® 




2 


83.9 ± 1.1 


1.30 X 10-12 




3 


72.8 ± 2.9 


2.35 X 10-s 




4 


81.0 ± 2.1 


1.35 X 10-1° 




5 


83.3 ± 1.2 


1.30 X 10-12 




Avg. 


0Z.3 ± D.o 


4.09 X iO 


4 


1 


74.7 ± 1.7 


2.82 X lO-'^ 




2 


92.3 ± 1.6 


1.36 X 10-20 




3 


81.5 ± 1.3 


3.07 X 10-11 




4 


80.5 ± 2.0 


1.35 X 10-10 




5 


83.5 ± 2.2 


1.30 X 10-12 




Avg. 


82.5 ± 6.4 


5.64 X 10-*^ 


5 


1 


82.7 ± 1.3 


6.55 X 10-12 




2 


86.3 ± 1.3 


6.56 X 10-15 




3 


78.9 ± 1.4 


2.17 X 10-9 




4 


82.2 ± 1.5 


6.55 X 10-12 




5 


81.0 ± 1.3 


1.35 X 10-10 




Avg. 


82.2 ± 2.7 


4.30 X 10-10 



Table 2: Offline performances represented as classification accuracies estimated using 10 runs of stratified 
10-fold CV. The classification accuracy is defined as the probability of correctly classifying a trial given 
the feature, f*, i.e. ^(corre ct |/*) = P(J|/* & T)P{I) + P{W\f* G W)P{W), where P(I|/* € I) and 
P{yV \f* G W) are defined in Online Calibration in the Methods section, and P{I) and P{W) are the prior 
probabilities of idling and walking class, respectively. 
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Day 1 



9 Hz 



F1 Fl 



FC3 FC1 fcr FC! FC4 



rr ca ci C7 u ce te 



CP3 CPI CP! cf^ 



PI Pil P3 



7^^ 

f pi 1^ Fp2 



Fl Fi f; " 

F« FC1 rC. FC! FCi 

IT CS C3 C1 C! Z2 C4 ce T! 

i^PC CP3 C*"* CPl CP! CPA „t 

TP? ^f^* TPS 

pj PI pz p: p4 



11 Hz 



Fl fi FZ " 

FC3 FC1 rCi FCS FC* FCJ "* 

U £7 CI Cl C; U CG TS 

CPS CP3 CPI CPI CP! „1 TP^ 

P« P3 P* ^ 



PM 



P02 



PCM. 



nr. 



13 Hz 

Fp1 Fp! 



•^=5 FCJ FC1 FCJ FC! F« 

Tf Ct C1 Cl ILl Dl T£ 

Tpj CPS CPS „s 

e, PI Pi P7 p. 



PM 



P02 



7^ 

Fpl Ppj Fp! 



Fl Fi FI 

FCS FCJ FC1 FCj FC? FC4 F« 

r? CS C3 C1 Ct C2 C< C< TS 

jpj CPS cpj 

PI f1 " PI 

PS Pfi 
P7 PB 

nrvi FOr 



Day 2 



Fl Fl f: 



FC1 FCl FCS 



u Ci c: ci ci cz CJ ce ts 

jpj CPS CP3 CPI CPl CPa CM cpj 

P3 " F? PI 



PQ3 



P03 



PCra 



Days 



H n fl " 

FC3 rci FCJ fCi FC4 FC* 

r7 CS Ci Ci C-l C« TS 

CPS CP3 CPI CP, CP! CPi j.p^ 

I f3 F3 Pi 



Fl FI Fi 



FC3 fci FC* F« FC4 



rr c^ C3 CI ci cz cj cs te 



CP3 Cf'l CPi CPi eP4 



PI Ps PJ 



Fl FI FJ 



FCJ fCl fCi FCI FC4 



r? CS C3 C-l Ci C2 C4 CC T£ 



CPJ CPI CPi CPi CM 



PI P^ Pi 



Day 4 



1 ^ n 



1 ^ f , 



Fl Fi Fa 



FCl FCi FC! FC4 



IT G5 C3 CI I 



CP3 CPl CPl CP! 0*4 



Fl ti FZ 



FCa FCl FCl FC2 FC* 



U CS CJ G1 Cl CZ CJ G6 T8 



CPS ffl CPi CPi CP* 



PI Pj pi 



FCJ FCl I^J FCJ FC4 



JJ Ci CJ 



I CZ c* ce ^s 



CP! CPI CPl CPS cpj 



PI Pi pj 



1 & F, 



A 

W Ni? Fp! 



Fl F^ FZ 



Day 5 



" M r, F! " 
f=S ICS [CI ft, FC! FM 
17 C) C1 Cl C3 C C« TS 



Fl fl FI 



FCl FCl fCl FCa FM 



rr Cj a CZ CZ C4 CE TE 



CPI CPI CPS 



P, PI Pi p; p4 



PS " " P= P4 



f*^' FCJ FCl FCl FC! FC4 



r; Ci C3 CI CE C2 C4 CC TE 



CPI CPI CP! 



P) " " " P« 



Figure 3: Feature extraction images of Subject 2 for all experimental sessions. Dark colors (red and blue) 
represent the areas that were responsible for encoding the differences between idling and walking KMI. The 
EEG power in the 9-13 Hz bins in the mid-frontal (FCz), central (Cz) and central-parietal (CPz) areas were 
the most salient. 
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Day 1 



9 Hz 



C2 C7 Ci CS 



Cf! CP! 



1 1 Hz 

FfI ^ Ft2 



AF4 



" f1 Ff F? " 
FC3 FC1 eCi FC2 FW FM 
Ci ^^^^ CI C7 ^^^B^ 



13 Hz 

Fp> ^ Fp? 



AF3 



FS „ Ffl 

" fl Fi f! " 

FCS FCd 

Clh C3 Ct Cl CI CA Ci 

fat CPi CP1 CPI CPi gpa i^e. 



Day 2 



1 ftf 
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Figure 4: Feature extraction images of 
Hz bins in the mid- frontal (Fz), lateral 
were the most informative for encodin; 



Subject 5 for all experimental sessions. The EEG power in the 9-13 
central (C3 and C4) and lateral central-parietal (CPS and CP4) areas 
I the differences between idling and walking KMI. 
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Online Parameters 



Subject 


Day 


Ti 




1 


1 


0.37 


0.47 




2 


0.42 


0.67 




3 


0.45 


0.55 




4 


0.22 


0.39 




5 


0.35 


0.44 


2 


1 


0.5 


0.5 




2 


0.07 


0.09 




3 


0.42 


0.45 




4 


0.6 


0.7 




5 


0.6 


0.66 


3 


1 


0.2 


0.3 




2 


0.57 


0.62 




3 


0.26 


0.75 




4 


0.7 


0.9 




5 


0.4 


0.9 


4 


1 


0.62 


0.8 




2 


0.61 


0.8 




3 


0.38 


0.45 




4 


0.4 


0.6 




5 


0.58 


0.65 


5 


1 


0.3 


0.85 




2 


0.4 


0.7 




3 


0.3 


0.7 




4 


0.4 


0.8 




5 


0.3 


0.85 



Table 3: The stop (and start) parameters, Tx (and Tyy), for online operation as determined by the calibration 
session. 
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k_/ V-4- \\-^\-' V 


Completion Time 


Siifcpssfnl StoDS 


Session 




meanzbstd (sec) 


meanzbstd 


Breakdown 


1 n=29 


275±45 


6.2±1.8 


(26 3) 


Best: Db-y 5 


298±77 


6.8±2.3 




Random Walk 


258±12 


6.9±1.3 




2 n=25 


271±66 


5.7±2.3 


(24 ^) 


Best: Day 5 


293±26 


8.1±1.2 




Random Walk 


1050±85 


10.0±0.2 




3 n=24 


277±65 


9.4±1.3 


f24 0) 


Bpst' T)e\v 4 


231±8 


10 0±0 




Random Walk 


>1200 


0.1±0.25 




4 n=19 


289±43 


8.3±1.8 


fl8 1) 


Best: Day 1 


264±12 


8.9±0.3 




Random Walk 


>1200 


4.3±0.7 




5 n=27 


258±31 


7.7±2.1 


(27, 0) 


Best: Day 4 


260±17 


lO.OiO.O 




Random Walk 


>1200 


5.1±1.4 




All subjects 


273±51 


7.4±2.3 


(119, 5) 


Physical joystick 


205.07±4.2 


9.38±0.95 





Table 4: Average online performances, including completion time and number of successful stops for each 
subject, upon achieving purposeful control, which was on day 2 for Subject 2, and on day 1 for all other 
subjects. Also presented are the average online performances of each subject's best experimental day. 
Performances of random walk are shown for comparison. Sessions are also broken down by number of 
purposeful and non-purposeful performances, i.e. (p<0.01, p>0.01) 
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Figure 6: Best online performances (completion times and stop scores) of each subject. Performances 
are marked by crosses with associated composite scores shown in parenthesis. PDF of the random walk 
simulations (contour lines) are also shown for Subject 1, but are absent for other subjects as the contours 
lie outside of the alloted 20-min limit. Note that all performances are purposeful (p<0.01). 



12 




100 120 
Time (sec) 



Figure 7: A representative time course of an online session where Subject 3 was able to achieve a high 
number of successful stops with a short completion time. False positives (i.e. the avatar walked when the 
subject intended to stop) are marked by orange segments, and false negatives (i.e. the avatar stopped when 
the subject intended to walk) are marked by red segments. 









Composite Score (%) 




Subject 




Day 1 


Day 2 


Day 3 


Day 4 


Day 5 


1 




66.2±3.1 


76.8±9.1 


75.0±11.3 


74.4±11.6 


76.9±11.9 




Best 


69.7 


90 


85.6 


80.5 


89.9 


2 






69.9±11.2 


69.3±13.5 


59.9±12.3 


85.4±5.4 




Best 




86.8 


89.3 


68.9 


90.4 


3 




86.4±3.2 


87.5±14.2 


93.0±5.6 


98.4±0.4 


97.3±2.2 




Best 


89.9 


95.2 


98.5 


99 


98.6 


4 




91.5±1.7 


90.1±5.3 


89.7±4.1 


79.4±16.3 


80.7±2.3 




Best 


93.2 


95.5 


93.1 


96.1 


82.3 


5 




66.4±9.0 


80.7±5.1 


93.5±3.5 


96.9±0.9 


88.7±6.1 




Best 


76.7 


86.8 


97.3 


98.2 


98.3 


Average 




77.8±13.0 


79.8±11.5 


81.7±13.9 


80.7±17.7 


85.7±10.2 


Joystick 








96.5±3.8 







Table 5: Average and best composite online performance score for all subjects for each experimental day 
as calculated using Eqs. [3] and |4j The composite scores for random walk and the physical joystick are also 
shown for comparison. Note that Subject 2 did not achieve purposeful control on day 1 and was therefore 
unable to participate in online sessions on this day. 
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Discussion 

The results of this study show that subjects with paraplegia or tetraplegia due to SCI can operate a non- 
invasive BCI-controUed avatar within a VRE to accomplish a goal-oriented ambulation task. All subjects 
gained purposeful online BCI control on the first day after undergoing a 10-min training session, with the 
exception of Subject 2, who did not attain control until day 2. In addition, BCI control was maintained 
and continued to improve over the course of the study. These findings suggest that a BCI-controlled lower 
extremity prosthesis for cither gait rehabilitation or restoration may be feasible. 

The offline classification accuracies varied across subjects and experimental days, but were significantly 
above the chance level performance (50%). Similar to able-bodied subjects engaged in the same task [8|[9], 
a short 10-min training session was sufficient for the data-driven machine learning algorithm to generate 
accurate subject-specific EEC decoding models for this population. The topographic maps of these models 
(e.g. Fig. 3 and Fig. 4) showed that the spatio-spectral features underlying the differences between walking 
and idling KMIs varied across subjects and evolved over experimental days. The differences in the brain areas 



and EEC frequencies across subjects may be due to variations in cortical reorganization following SCI 10 - 13 



or due to differences in imageries employed by each subject (e.g. the KMI of walking instructions may have 
been interpreted differently by each subject). Nevertheless, all subjects showed activation of mid- frontal 
areas, which likely overlay the pre- motor and supplementary motor areas, as well as the pre- frontal cortex. 
Their activation during walking KMI is consistent with functional imaging findings, such as those in [24) . 
Another common pattern across subjects was the presence of activity near bilateral, lateral central-parietal 
electrodes, which likely represents the arm sensorimotor areas. A similar pattern was observed in able-bodied 
individuals '9j , and is hypothesized to originate from arm swing imagery. Finally, the evolution of the feature 
extraction maps over the 5 experimental days may be indicative of a neuro-plasticity process associated with 



practice and learning 25 26 



The spatio-temporal variations of walking KMI activation patterns demonstrate the necessity of a data- 
driven machine learning approach for rapid acquisition of intuitive BCI control. First, our approach accom- 
modates for the variations of these activity patterns across subjects, as well as their evolution over time. 
Second, it facilitates rapid acquisition of online BCI control, presumably by enabling subjects to utilize 
intuitive mental strategies. The user training time necessary to acquire purposeful BCI control in this study 
is significantly shorter than those of other BCI studies where users must learn a completely new cognitive 



skill to modulate pre-selected EEC features, such as the /i-rhythm over lateral central areas 27 . Finally, 



this approach carries a significant potential value in the future practical implementation of BCI-prostheses, 
as it may drastically reduce the training time needed to attain purposeful and useful BCI control from a 
timescale of weeks to months to one of minutes to days. This in turn may significantly reduce the cost of 
training users to operate future BCI-prostheses. 

The results presented in Table 4 show that once purposeful control was achieved, it was maintained in 
96% of all online sessions. In addition, 3 out of 5 subjects achieved successful stop scores similar to those 
obtained using a manually controlled joystick. Even though no subjects were able to complete the course 
as fast as manual control, it is encouraging that the average composite scores increased significantly over 
the course of the study. Furthermore, the average composite scores (Table 5) improved over time, with the 
best scores approaching 100% for Subjects 3 and 5 by the end of the study. Therefore, not only was online 
control significantly different from random walk, but it was also meaningful. Given this trend, additional 
training and practice may help further improve performance, possibly to the point of approaching that of 
the manually controlled joystick. In conclusion, the high level of online control achieved by SCI subjects 
over the course of 5 experimental days suggests that it may be feasible to apply this BCI system to control a 
lower extremity prosthesis for ambulation after SCI. Furthermore, the proposed BCI- VRE system may serve 
as a training platform for operation of BCI lower extremity prostheses once they become widely available. 



Conclusions 

This study shows that SCI subjects can purposefully operate a self-paced BCI- VRE system in real time, 
and that the current BCI design approach is able to overcome the potential problems associated with 
variations in neurophysiology due to cortical reorganization after SCI, learning and plasticity processes, 
and differences in KMI strategies. Furthermore, the system satisfies the requirements of an ideal BCT 
lower extremity prosthesis set forth in [o], namely: intuitiveness, robustness, and short training time. The 
operation of the system is intuitive as it enabled subjects to use walking KMI to control the ambulation 
of the avatar. The system is robust in that the data-driven decoding methodology was able to successfully 
accommodate for subject-to-subject and day-to-day variations in the neurophysiological underpinnings of 
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idling and walking KMI behaviors. In addition, subjects were able to maintain purposeful online control 
over the course of several weeks, further underscoring the system's robustness over time. Finally, the system 
required only a short training time, as BCI control was generally attained after only a 10-min training data 
collection procedure followed by a 2-min calibration session on the I''' experimental day (for 4 out of the 5 
subjects). The successful outcome of this study indicates that BCI-controUed lower extremity prostheses for 
gait rehabilitation or restoration after SCI may be feasible in the future. 
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